Dictionary Learning Algorithms and Applications by Bogdan Dumitrescu & Paul Irofti

Author:Bogdan Dumitrescu & Paul Irofti
Language: eng
Format: epub
Publisher: Springer International Publishing, Cham

Example 5.12

We compare Algorithms 5.3 and 5.4 on a simple DL problem with m = 16, n = 40, N = 1000, s = 4. The data are generated synthetically, with a random dictionary and an SNR of 20 dB. The algorithms are run for more than N iterations by circularly feeding them with the signals. The RLS-DL in the form of Algorithm 5.4 is indeed unstable and stops converging after 2500–3000 iterations; so, we report the results of the algorithm modified as presented in Remark 5.11. Figure 5.1 shows the evolution of the RMSE for the two algorithms for two forgetting factors, φ = 0.995 and φ = 0.999. The RMSE is averaged over ten runs with different data; in each run, both algorithms are initialized with the same random dictionary. For computing the RMSE, we use the definition (2.​10), which is appropriate since the signals are used several times, hence we deal actually with a standard DL problem (not one in which the model may be indeed time variant). Note that computing the RMSE in such a way at each iteration is extremely expensive with respect to the complexity of the algorithm and is done here only for examination purposes. A first remark is that the convergence speed of RLS-DL is almost insensitive to the value of the forgetting factor. On the contrary, the coordinate descent algorithm is affected significantly when the forgetting factor is large. However, the RMSE attained by coordinate descent is lower, in these experiments. Although the results seem to give a small advantage to coordinate descent, we must not forget that RLS-DL has a much lower complexity and that it is also more robust with respect to the forgetting factor values.

Fig. 5.1RMSE given by online DL algorithms based on coordinate descent (blue) and RLS (red) for a forgetting factor φ = 0.995 (up) and φ = 0.999 (down)

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.